Comparison between random forests, artificial neural networks and gradient boosted machines methods of on-line vis-NIR spectroscopy measurements of soil total nitrogen and total carbon

Accurate and detailed spatial soil information about within-field variability is essential for variable-rate applications of farm resources. Soil total nitrogen (TN) and total carbon (TC) are important fertility parameters that can be measured with on-line (mobile) visible and near infrared (vis-NIR) spectroscopy. This study compares the performance of local farm scale calibrations with those based on the spiking of selected local samples from both fields into an European dataset for TN and TC estimation using three modelling techniques, namely gradient boosted machines (GBM), artificial neural networks (ANNs) and random forests (RF). The on-line measurements were carried out using a mobile, fiber type, vis-NIR spectrophotometer (305-2200 nm) (AgroSpec from tec5, Germany), during which soil spectra were recorded in diffuse reflectance mode from two fields in the UK. After spectra pre-processing, the entire datasets were then divided into calibration (75%) and prediction (25%) sets, and calibration models for TN and TC were developed using GBM, ANN and RF with leave-one-out cross-validation. Results of cross-validation showed that the effect of spiking of local samples collected from a field into an European dataset when combined with RF has resulted in the highest coefficients of determination (R-2) values of 0.97 and 0.98, the lowest root mean square error (RMSE) of 0.01% and 0.10%, and the highest residual prediction deviations (RPD) of 5.58 and 7.54, for TN and TC, respectively. Results for laboratory and on-line predictions generally followed the same trend as for cross-validation in one field, where the spiked European dataset-based RF calibration models outperformed the corresponding GBM and ANN models. In the second field ANN has replaced RF in being the best performing. However, the local field calibrations provided lower R-2 and RPD in most cases. Therefore, from a cost-effective point of view, it is recommended to adopt the spiked European dataset-based RF/ANN calibration models for successful prediction of TN and TC under on-line measurement conditions

Estimation of secondary soil properties by fusion of laboratory and on-line measured vis-NIR spectra

Visible and near infrared (vis-NIR) diffuse reflectance spectroscopy has made invaluable contributions to the accurate estimation of soil properties having direct and indirect spectral responses in NIR spectroscopy with measurements made in laboratory, in situ or using on-line (while the sensor is moving) platforms. Measurement accuracies vary with measurement type, for example, accuracy is higher for laboratory than on-line modes. On-line measurement accuracy deteriorates further for secondary (having indirect spectral response) soil properties. Therefore, the aim of this study is to improve on-line measurement accuracy of secondary properties by fusion of laboratory and on-line scanned spectra. Six arable fields were scanned using an on-line sensing platform coupled with a vis-NIR spectrophotometer (CompactSpec by Tec5 Technology for spectroscopy, Germany), with a spectral range of 305-1700 nm. A total of 138 soil samples were collected and used to develop five calibration models: (i) standard, using 100 laboratory scanned samples; (ii) hybrid-1, using 75 laboratory and 25 on-line samples; (iii) hybrid-2, using 50 laboratory and 50 on-line samples; (iv) hybrid-3, using 25 laboratory and 75 on-line samples, and (v) real-time using 100 on-line samples. Partial least squares regression (PLSR) models were developed for soil pH, available potassium (K), magnesium (Mg), calcium (Ca), and sodium (Na) and quality of models were validated using an independent prediction dataset (38 samples). Validation results showed that the standard models with laboratory scanned spectra provided poor to moderate accuracy for on-line prediction, and the hybrid-3 and real-time models provided the best prediction results, although hybrid-2 model with 50% on-line spectra provided equally good results for all properties except for pH and Na. These results suggest that either the real-time model with exclusively on-line spectra or the hybrid model with fusion up to 50% (except for pH and Na) and 75% on-line scanned spectra allows significant improvement of on-line prediction accuracy for secondary soil properties using vis-NIR spectroscopy

Development of a framework for the evaluation of the environmental benefits of controlled traffic farming

Although controlled traffic farming (CTF) is an environmentally friendly soil management system, no quantitative evaluation of environmental benefits is available. This paper aims at establishing a framework for quantitative evaluation of the environmental benefits of CTF, considering a list of environmental benefits, namely, reducing soil compaction, runoff/erosion, energy requirement and greenhouse gas emission (GHG), conserving organic matter, enhancing soil biodiversity and fertiliser use efficiency. Based on a comprehensive literature review and the European Commission Soil Framework Directive, the choice of and the weighting of the impact of each of the environmental benefits were made. The framework was validated using data from three selected farms. For Colworth farm (Unilever, UK), the framework predicted the largest overall environmental benefit of 59.3% of the theoretically maximum achievable benefits (100%), as compared to the other two farms in Scotland (52%) and Australia (47.3%). This overall benefit could be broken down into: reducing soil compaction (24%), tillage energy requirement (10%) and GHG emissions (3%), enhancing soil biodiversity (7%) and erosion control (6%), conserving organic matter (6%), and improving fertiliser use efficiency (3%). Similar evaluation can be performed for any farm worldwide, providing that data on soil properties, topography, machinery, and weather are available

Influence of the number of samples on prediction error of visible and near infrared spectroscopy of selected soil properties at the farm scale.

Although visible and near infrared (vis-NIR) spectroscopy has proved to be a fast, inexpensive and relatively accurate tool to measure soil properties, considerable research is required to optimise the calibration procedure and establish robust calibration models. This paper reports on the influence of the number of samples used for the development of farm-scale calibration models for moisture content (MC), total nitrogen (TN) and organic carbon (OC) on the prediction error expressed as root mean square error of prediction (RMSEP). Fresh (wet) soil samples collected from four farms in Czech Republic, Germany, Denmark and the UK were scanned with a fibre type vis-NIR, AgroSpec spectrophotometer (tec5 Technology for Spectroscopy, Germany) with a spectral range of 305 - 2200 nm. Spectra were divided into calibration (two-third) and prediction (one-third) sets and the calibration spectra were subjected to a partial least squares regression (PLSR) with leave-one-out cross validation using Unscrambler 7.8 software (Camo Inc., Oslo, Norway). The RMSEP values of models with large sample number (46 - 84 samples from each farm) were compared with those of models developed using small sample number (25 samples selected from the large sample set of each farm) for the same variation range. Both large set and small set models were validated by the same prediction set for each property. Further PLSR analysis was carried out on samples from the German farm, with different sample number of the calibration set of 25, 50, 75 and 100 samples. Results showed that the large-size dataset models resulted in lower RMSEP values than the small-size dataset models for all the soil properties studied. The results also demonstrated that with the increase in sample number used in the calibration set, RMSEP decreased in almost linear fashion, although the largest decrease was between 25 and 50 samples. Therefore, it is recommended to chose the number of samples according to accuracy required, although 50 soil samples is considered appropriate in this study to establish calibration models of TN, OC and MC with smaller expected prediction errors as compared with smaller sample numbers

Glucose adulteration in Saudi honey with visible and near infrared spectroscopy

This article reports on the implementation of visible and near infrared spectroscopy for the detection of glucose concentration in a mixture of Saudi and imported honey samples adulterated by glucose syrup of five concentrations: 0, 5, 12, 19, and 33 g/100 g. Honey samples were scanned in trans-reflectance mode with an AgroSpec mobile, fibre type, visible and near infrared spectrophotometer (tec5 Technology for Spectroscopy, Germany), with a measurement range of 305–2200 nm. The entire data set of 345 spectra was randomly divided into calibration (70%) and prediction (30%) sets. The first group was subjected to a partial least squares regression analysis with a leave-one-out cross-validation to establish a calibration model for the prediction of glucose concentration, whereas the second group was used to validate the partial least squares model. For the cross-validation, the values for root mean square error of prediction, coefficient of determination, and ratio of prediction deviation, which is the standard deviation divided by root mean square error of prediction were 4.52 g/100 g, 0.85, and 2.53, respectively. A slightly lower range of accuracy was obtained in the prediction set, with root mean square error of prediction, coefficient of determination, and ratio of prediction deviation values of 5.56 g/100 g, 0.78 and 2.06, respectively. The results achieved suggest that the visible and near infrared spectroscopy is a powerful technique for the quantification of glucose adulteration in Saudi honey

On-line Vis-Nir sensor determination of soil variations of sodium, potassium and magnesium

Among proximal measurement methods, visible and near infrared (Vis-Nir) spectroscopy probably has the greatest potential for determining the physico-chemical properties of different natural resources, including soils. This study was conducted to determine the sodium, potassium and magnesium variations in a 10. Ha field located in Karacabey district (Bursa Province, Turkey) using an on-line Vis-Nir sensor. A total of 92 soil samples were collected from the field. The performance and accuracy of the Na, K and Mg calibration models was evaluated in cross-validation and independent validation. Three categories of maps were developed: 1) reference laboratory analyses maps based on 92 points 2) Full-data point maps based on all 6486 on-line points Vis-Nir predicted in 2013 and 3) full- data point maps based on all 2496 on-line points Vis-Nir predicted in 2015. Results showed that the prediction performance in the validation set was successful, with average R2 values of 0.82 for Na, 0.70 for K, and 0.79 for Mg, average root mean square error of prediction (RMSEP) values of 0.02% (Na), 0.20% (K), and 1.32% (Mg) and average residual prediction deviation (RPD) values of 2.13 (Na), 0.97 (K), and 2.20 (Mg). On-line field measurement was also proven to be successful with validation results showing average R2 values of 0.78 (Na), 0.64 (K), and 0.60 (Mg), average RMSEP values of 0.04% (Na), 0.13% (K), and 2.19% (Mg) and average RPD values of 1.57 (Na) 1.68 (K) and 1.56 (Mg). Based on 3297 points, maps of Na, K and Mg were produced after N, P, K and organic fertilizer applications, and these maps were then compared to the corresponding maps from the previous year. The comparison showed a variation in soil properties that was attributed to the variable rate of fertilization implemented in the preceding year